from IPython.display import HTML
HTML('''<script>
code_show=true;
function code_toggle() {
if (code_show){
$('div.input').hide();
} else {
$('div.input').show();
}
code_show = !code_show
}
$( document ).ready(code_toggle);
</script>
For a more presentable and an easy reading experience, the raw code for this iPython notebook is hidden.
If you want to see the raw code, please click <a href="javascript:code_toggle()">here</a>.''')
from IPython.display import Image
Image("images/skin_disease_classes.png")
Most of the images above do look alike. So how do we confirm which is what then? Do you need to consult a doctor immediately?
In this mini project, I will design an algorithm that can visually diagnose melanoma, the deadliest form of skin cancer. In particular, my algorithm will distinguish this malignant skin tumor from two types of benign lesions (nevi and seborrheic keratoses).
The data and objective are pulled from the __2017 ISIC Challenge on Skin Lesion Analysis Towards Melanoma Detection__. As part of the challenge, participants were tasked to design an algorithm to diagnose skin lesion images as one of three different skin diseases (melanoma, nevus, or seborrheic keratosis). In this project, I will create a model to generate my own predictions using my developed model.
from IPython.display import Image
Image("images/AI_north_qld.png")
There are characteristics that doctors use when identifying cancerous skin lesions e.g. fuzziness of the border, the assymmetry, the coloration, the growth rate if it is accessible.
Undoubtedly, trained human eye can be rather good at finding skin spots that maybe cancerous. but there are times some best doctors could miss some prominent cases. Hence, this process takes in an artificial intelligence (AI) algo in detecting skin diseases.
Wehat that means is we can, now, use our own computer where it takes in the full-body photos of a patient's skin, then sends the high-resolution photos through a neural network model (AI) to denote the type of skin disease and if the risk of any skin spots being malignant.
To ensure we get a good diagnosis on the skin image, we need to ensure our AI model has a good sound accuracy rate.
from IPython.display import YouTubeVideo
YouTubeVideo('6N-_lO-SXz0', width = 800, height = 500)
from IPython.display import Image
Image("images/skin_lesion.png")
Note, this is also known as: Skin cancer, malignant melanoma.
What is melanoma?
Melanoma is a type of skin cancer which usually occurs on the parts of the body that have been overexposed to the sun. Rare melanomas can occur inside the eye or in parts of the skin or body that have never been exposed to the sun.
Melanoma is projected to be the third most common cancer diagnosed in Australia in 2020, which along with New Zealand has the world's highest incidence rate for melanoma. Melanoma is more commonly diagnosed in men than women. The risk of being diagnosed with melanoma by age 85 is 1 in 13 for men compared to 1 in 21 for women.
It is estimated that 16,221 new cases of melanoma will be diagnosed in Australia in 2020.
Melanoma symptoms
Often melanoma has no symptoms, however, the first sign is generally a change in an existing mole or the appearance of a new spot. These changes can include:
New moles and spots will appear and change during childhood, adolescence and during pregnancy and this is normal. However, adults who develop new spots or moles should have them examined by their doctor.
Causes of melanoma
Melanoma risk increases with exposure to UV radiation from the sun or other sources such as solariums, particularly with episodes of sunburn (especially during childhood).
Melanoma risk is increased for people who have:
Diagnosis of melanoma
Melanoma can vary in the way it looks. The first sign is usually a new spot or change in an existing mole.
Physical examination
If you do notice any changes to your skin, your doctor will examine you and carefully check any spots you have identified as changed. Your doctor will use a handheld magnifying instrument (dermascope) and consider the criteria known as “ABCDE”. Further tests may be carried out by your GP or you may be referred to a specialist (dermatologist).
Biopsy
If the doctor suspects that a spot on your skin could be melanoma, an excision biopsy is carried out with the removal of the whole spot. This will then be examined under a microscope by a specialist to see if there are any cancer cells.
Checking lymph nodes
Your doctor may feel the lymph nodes near the melanoma to see if they are enlarged as melanoma can sometimes travel via the lymph vessels to other parts of your body. Your doctor may also recommend a biopsy to take a sample of the cells from an enlarged lymph node for further examination under a microscope.
If the doctor suspects melanoma, a biopsy may be carried out. This may be done by your GP or you may be referred to another specialist.
After a diagnosis of melanoma
After being diagnosed with melanoma, you may feel shocked, upset, anxious or confused. These are normal responses. A diagnosis of melanoma affects each person differently. For most it will be a difficult time, however some people manage to continue with their normal daily activities.
What is a nevus?
The word nevus indicates a benign (non-cancerous) skin or mucosal lesion comprising an abnormal mixture of a tissue’s normal components, usually presenting at birth or at a young age. A nevus is a congenital (present at birth) or acquired growths or pigmented blemishes on the skin; birthmarks or moles. As an example, mole is a melanocytic nevus. A nevus may also form from other skin cells (e.g., vascular nevus, which are formed from blood vessels). Some of these are also congenital (present at birth).
Nevus symptoms
Intradermal nevi appear as flesh-colored bumps on the surface of the skin, though they can also appear slightly brown. In some cases, they’ll contain brown spots of small dilated blood vessels.
Causes of nevus
An intradermal nevus is the result of one of three causes:
Diagnosis of nevus
Unless your mole has recently changed in size, shape, or color, no treatment is necessary for an intradermal nevus. However, it is possible to remove the mole if that’s what you’d like.
This also known as SK.
What is a seborrheic keratoses?
A seborrheic keratosis (seb-o-REE-ik ker-uh-TOE-sis) is a common noncancerous skin growth. People tend to get more of them as they get older.Seborrheic keratoses are usually brown, black or light tan. The growths look waxy, scaly and slightly raised. They usually appear on the head, neck, chest or back.Seborrheic keratoses are harmless and not contagious.
Seborrheic Keratoses symptoms
A seborrheic keratosis usually looks like a waxy or wartlike growth. It typically appears on the face, chest, shoulders or back. You may develop a single growth, though multiple growths are more common.
A seborrheic keratosis:
Causes of seborrheic keratoses
Doctors don't know exactly what causes seborrheic keratoses. The growths tend to run in some families, so genes may play a role.
Diagnosis of seborrheic keratoses
Diagnosis can be made by physical examination or skin biopsy.
from IPython.display import Image
#Image("images/cartoon.png")
Image("images/machinelearning.gif")
<IPython.core.display.Image object>
#import ipywidgets as widgets
#import IPython.display as display
## Read images from file (because this is binary, maybe you can find how to use ByteIO) but this is more easy
#img1 = open('images/icon.png', 'rb').read()
#img2 = open('images/sklearn.png', 'rb').read()
## Create image widgets. You can use layout of ipywidgets only with widgets.
## Set image variable, image format and dimension.
#wi1 = widgets.Image(value=img1, format='png', width=450, height=300)
#wi2 = widgets.Image(value=img2, format='png', width=450, height=300)
## Side by side thanks to HBox widgets
#sidebyside = widgets.HBox([wi1, wi2])
## Finally, show.
#display.display(sidebyside)
Before we get started, let's bring in all the required data tools and models for this project. On normal circumstances, I like to break this mechanics into two sub-parts:
# Import basic libraries
import matplotlib.pyplot as plt
import numpy as np # for linear algebra eg np.log, np.where, np.mean, np.std
import pandas as pd # for import data eg pd.read_csv, pd.DataFrame or pd.Series, pd.get_dummies(col_name,drop_first=True)
import seaborn as sns # mainly for visualisation
#import plotly.express as px #plotly visualisation (aka ggplot2 in R)
#from tabulate import tabulate #construct table
from datetime import datetime# if dealing with date and time or time series data
sns.set() # Optional:: this just make the plot nicer by changing the color, fontsize etc
#This syntax helps to display inline within frontends like the Jupyter notebook,
#directly below the code cell that produced it.
#The resulting plots will published nicedly within the notebook document.
%matplotlib inline
#display all float to 3dp
pd.set_option('display.float','{:.3f}'.format)
pd.set_option('display.max_columns',50)
pd.set_option('display.max_rows',100)
#Warning messages are typically issued in situations where it is useful to alert the user of some condition in a program,
#where that condition (normally) doesn’t warrant raising an exception and terminating the program.
#For example, one might want to issue a warning when a program uses an obsolete module.
from warnings import filterwarnings
filterwarnings('ignore')
%matplotlib inline
import csv
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
from glob import glob
from collections import Counter
import os
import cv2
from sklearn.datasets import load_files
from PIL import Image, ImageFont
from tqdm import tqdm
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision.models as models
import torchvision.transforms as transforms
import torchvision.datasets as datasets
import torch.optim as optim
import keras
from keras.preprocessing import image
from keras.preprocessing.image import ImageDataGenerator, img_to_array
from keras.utils import np_utils
Now, let's check out a few things before we get to the real stuff going.
# check if folder exists
print('There are 3 folders created for this project. We need to ensure each folder exists and accesible.')
print('Train dataset folder is set to', os.path.exists('./data/Train'))
print('Valid dataset folder is set to', os.path.exists('./data/Valid'))
print('Test dataset folder is set to', os.path.exists('./data/Test'))
print('Note: True means accessible. False means folder could not be found.')
There are 3 folders created for this project. We need to ensure each folder exists and accesible. Train dataset folder is set to True Valid dataset folder is set to True Test dataset folder is set to True Note: True means accessible. False means folder could not be found.
# load filenames for human, dog and cat images
print('Next, lets check how many images are there in each folder....\n')
train = np.array(glob("./data/train/*/*"))
valid = np.array(glob("./data/valid/*/*"))
test = np.array(glob("./data/test/*/*"))
# print number of images in each dataset
print('There are %d total train images.' % len(train))
print('There are %d total valid images.' % len(valid))
print('There are %d total test images.' % len(test))
print('\nIn summary, there are a total of %d images.' % (len(train)+len(valid)+len(test)))
Next, lets check how many images are there in each folder.... There are 2000 total train images. There are 150 total valid images. There are 600 total test images. In summary, there are a total of 2750 images.
The breakdown on each folder is as follows:
train_folder = np.array(glob("./data/train/*"))
valid_folder= np.array(glob("./data/valid/*"))
test_folder = np.array(glob("./data/test/*"))
for filename in np.hstack((train_folder, valid_folder, test_folder)):
print(filename + 'has:{} images.'.format(len(np.array(glob(filename+'\*')))))
./data/train\melanomahas:374 images. ./data/train\nevushas:1372 images. ./data/train\seborrheic_keratosishas:254 images. ./data/valid\melanomahas:30 images. ./data/valid\nevushas:78 images. ./data/valid\seborrheic_keratosishas:42 images. ./data/test\melanomahas:117 images. ./data/test\nevushas:393 images. ./data/test\seborrheic_keratosishas:90 images.
def display_image(img_path, title="Title"):
image = Image.open(img_path)
plt.title(title)
plt.imshow(image)
plt.grid(None)
#plt.show()
# obtain one batch of training images
#dataiter = iter(train_loader)
#images, labels = dataiter.next()
# plot the images in the batch, along with the corresponding labels
#fig = plt.figure(figsize=(25, 4))
#plot_size=20
#for idx in np.arange(plot_size):
# ax = fig.add_subplot(2, plot_size/2, idx+1, xticks=[], yticks=[])
# ax.imshow(np.transpose(images[idx], (1, 2, 0)))
# # print out the correct label for each image
# # .item() gets the value contained in a Tensor
# ax.set_title(str(labels[idx].item()))
from PIL import Image
fig = plt.figure(figsize=(18,18))
fig.subplots_adjust(wspace=0.2, hspace=0.4)
plt.suptitle('Samples || Ref: Type of Skin Diseases\Image_File#', fontsize=20)
# samples some images
k = 10 #start with kth image
n = 6 # number of images required in each skin diseases #can go in a range of [2,3,4,5,6]
i = 0 # counter
for filename in np.hstack((np.array(glob("./data/train/melanoma/*")[k:k+n]),np.array(glob("./data/train/nevus/*")[k:k+n]),np.array(glob("./data/train/seborrheic_keratosis/*")[k:k+n]))):
i += 1
ax = fig.add_subplot(n,n,i)
image = Image.open(filename)
plt.title(filename.rsplit(".",1)[0].replace('./data/train/','').replace('ISIC_',''))
ax.imshow(image)
ax.set_axis_off()
ax.grid(None)
#from PIL import Image
#fig = plt.figure(figsize=(5,5))
#fig.subplots_adjust(wspace=0.2, hspace=0.4)
#for filename in np.hstack((np.array(glob("./data/train/melanoma/*"))[:3],np.array(glob("./data/train/nevus/*"))[:3],np.array(glob("./data/train/seborrheic_keratosis/*"))[:3])):
# image = Image.open(filename)
# fig = plt.figure(figsize=(3,3))
# #plt.subplot(3,3,i)
# plt.title(filename.rsplit(".",1)[0].replace('./data/train/','').replace('ISIC_',''))
# plt.imshow(image)
# plt.grid(None)
# plt.show()
Data loading is one of the first steps in building a Deep Learning pipeline, or training a model. This task becomes more challenging when the complexity of the data increases.
In this section, we will learn about the DataLoader class in PyTorch that helps us to load and iterate over elements in a dataset. This class is available as DataLoader in the torch.utils.data module.
Dataset: The first parameter in the DataLoader class is the dataset. This is where we load the data from.
Batching the data: batch_size refers to the number of training samples used in one iteration. Usually we split our data into training and testing sets, and we may have different batch sizes for each.
Shuffling the data: shuffle is another argument passed to the DataLoader class. The argument takes in a Boolean value (True/False). If shuffle is set to True, then all the samples are shuffled and loaded in batches. Otherwise they are sent one-by-one without any shuffling.
Allowing multi-processing: As deep learning involves training models with a lot of data, running only single processes ends up taking a lot of time. In PyTorch, you can increase the number of processes running simultaneously by allowing multiprocessing with the argument num_workers. This also depends on the batch size, but I wouldn’t set num_workers to the same number because each worker loads a single batch, and returns it only once it’s ready.
Merging datasets: The collate_fn argument is used if we want to merge datasets. This argument is optional, and mostly used when batches are loaded from map-styled datasets.
Loading data on CUDA tensors: You can directly load datasets as CUDA tensors using the pin_memory argument. It is an optional parameter that takes in a Boolean value; if set to True, the DataLoader class copies Tensors into CUDA-pinned memory before returning them.
Note:
Batch size is one of the most important hyperparameters to tune in deep learning. I prefer to use a larger batch size to train my models as it allows computational speedups from the parallelism of GPUs. However, it is well known that too large of a batch size will lead to poor generalization. On the one extreme, using a batch equal to the entire dataset guarantees convergence to the global optima of the objective function. However this is at the cost of slower convergence to that optima. On the other hand, using smaller batch sizes have been shown to have faster convergence to good results. This is intuitively explained by the fact that smaller batch sizes allow the model to start learning before having to see all the data. The downside of using a smaller batch size is that the model is not guaranteed to converge to the global optima.Therefore it is often advised that one starts at a small batch size reaping the benefits of faster training dynamics and steadily grows the batch size through training.
from PIL import Image
batch_size = 20
num_workers = 0
train_transforms = transforms.Compose([transforms.RandomRotation(degrees=(-10, 10),resample=False,expand=False),
transforms.RandomHorizontalFlip(p=0.5),
transforms.RandomResizedCrop(size=(224, 224),scale=(0.08, 1.0),ratio=(0.75, 1.3333),interpolation=Image.BILINEAR),
transforms.Resize(size=(255, 255), interpolation=Image.BILINEAR),
transforms.CenterCrop(size=(256, 256)),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])
valid_transforms = transforms.Compose([transforms.RandomRotation(degrees=(-10, 10),resample=False,expand=False),
transforms.RandomHorizontalFlip(p=0.5),
transforms.RandomResizedCrop(size=(224, 224),scale=(0.08, 1.0),ratio=(0.75, 1.3333),interpolation=Image.BILINEAR),
transforms.Resize(size=(255, 255), interpolation=Image.BILINEAR),
transforms.CenterCrop(size=(256, 256)),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])
test_transforms = transforms.Compose([transforms.Resize(size=(255, 255), interpolation=Image.BILINEAR),
transforms.CenterCrop(size=(256, 256)),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])
# data directory / datasets
data_dir = './data'
train_data = datasets.ImageFolder(os.path.join(data_dir, 'train/'), transform=train_transforms)
valid_data = datasets.ImageFolder(os.path.join(data_dir, 'valid/'), transform=valid_transforms)
test_data = datasets.ImageFolder(os.path.join(data_dir, 'test/'), transform=train_transforms)
#defining the data loaders
# the data is splitted between training, testing and validation sets
# data loader
train_loader = torch.utils.data.DataLoader(train_data, batch_size=batch_size, num_workers=num_workers, shuffle=True)
valid_loader = torch.utils.data.DataLoader(valid_data, batch_size=batch_size, num_workers=num_workers, shuffle=True)
test_loader = torch.utils.data.DataLoader(test_data, batch_size=batch_size, num_workers=num_workers, shuffle=True)
data_loaders = {'train': train_loader,'valid': valid_loader,'test': test_loader}
print(data_loaders)
{'train': <torch.utils.data.dataloader.DataLoader object at 0x0000022B877861F0>, 'valid': <torch.utils.data.dataloader.DataLoader object at 0x0000022B9A4045E0>, 'test': <torch.utils.data.dataloader.DataLoader object at 0x0000022B9A4042B0>}
# print out some data stats
print('loaders_scratch: ', data_loaders)
print('Num training images: ', len(train_data))
print('Num valid images: ', len(valid_data))
print('Num test images: ', len(test_data))
print('\n')
print('Train transform: \n ', train_transforms)
print('\n')
print('Validation transform: \n ', valid_transforms)
print('\n')
print('Test transform: \n ', test_transforms)
loaders_scratch: {'train': <torch.utils.data.dataloader.DataLoader object at 0x0000022B877861F0>, 'valid': <torch.utils.data.dataloader.DataLoader object at 0x0000022B9A4045E0>, 'test': <torch.utils.data.dataloader.DataLoader object at 0x0000022B9A4042B0>}
Num training images: 2000
Num valid images: 150
Num test images: 600
Train transform:
Compose(
RandomRotation(degrees=(-10, 10), resample=False, expand=False)
RandomHorizontalFlip(p=0.5)
RandomResizedCrop(size=(224, 224), scale=(0.08, 1.0), ratio=(0.75, 1.3333), interpolation=PIL.Image.BILINEAR)
Resize(size=(255, 255), interpolation=PIL.Image.BILINEAR)
CenterCrop(size=(256, 256))
ToTensor()
Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
)
Validation transform:
Compose(
RandomRotation(degrees=(-10, 10), resample=False, expand=False)
RandomHorizontalFlip(p=0.5)
RandomResizedCrop(size=(224, 224), scale=(0.08, 1.0), ratio=(0.75, 1.3333), interpolation=PIL.Image.BILINEAR)
Resize(size=(255, 255), interpolation=PIL.Image.BILINEAR)
CenterCrop(size=(256, 256))
ToTensor()
Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
)
Test transform:
Compose(
Resize(size=(255, 255), interpolation=PIL.Image.BILINEAR)
CenterCrop(size=(256, 256))
ToTensor()
Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
)
- transforms.Resize() to resize images,
- transforms.CenterCrop() to crop the images from the center,
- transforms.RandomResizedCrop() to randomly resize all the images,
- transforms.ToTensor() to convert the images to the Tensor datatype,
- transforms.Normalize(mean,std) normalizes all the values in the tensor so that they lie between 0.5 and 1.
dataiter = iter(test_loader)
images, labels = dataiter.next()
print(type(images))
print(images.shape)
print(labels.shape)
print('The images consists of 224 X 224 pixel with 3 colour channels')
<class 'torch.Tensor'> torch.Size([20, 3, 256, 256]) torch.Size([20]) The images consists of 224 X 224 pixel with 3 colour channels
num_skins = len(train_data.class_to_idx)
print('Total number of skin cancers in the train dataset are {}.'.format(num_skins))
class_names = [item[:].replace("_", " ") for item in data_loaders['train'].dataset.classes]
print('Samples of difference classes are:',class_names)
Total number of skin cancers in the train dataset are 3. Samples of difference classes are: ['melanoma', 'nevus', 'seborrheic keratosis']
import torchvision
print('Image Viewing Process:')
# helper imshow function
def imshow(img):
npimg = img.detach().numpy()
img = np.transpose(npimg, (1, 2, 0))
img = ((img + 1)*255 / (2)).astype(np.uint8)
ax.xaxis.set_visible(False)
ax.yaxis.set_visible(False)
im = ax.imshow(img.reshape((256,256,3)))
plt.imshow(img)
# get some images from X
dataiter = iter(test_loader)
# the "_" is a placeholder for no labels
images, _ = dataiter.next()
# show images
fig = plt.figure(figsize=(12, 8))
imshow(torchvision.utils.make_grid(images))
Image Viewing Process:
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-13-47581cc1052c> in <module> 18 # show images 19 fig = plt.figure(figsize=(12, 8)) ---> 20 imshow(torchvision.utils.make_grid(images)) <ipython-input-13-47581cc1052c> in imshow(img) 8 ax.xaxis.set_visible(False) 9 ax.yaxis.set_visible(False) ---> 10 im = ax.imshow(img.reshape((256,256,3))) 11 plt.imshow(img) 12 ValueError: cannot reshape array of size 4809648 into shape (256,256,3)
<Figure size 864x576 with 0 Axes>
To simplify the model architecture, I am using transfer learning from a pre-trained model to create a CNN that can identify type of cancer from images. As we are dealing with images which has higher pixel resolutions, it means the processing is likely to take up days if we running this on our local computers. As such, it is highly suggested that we leverage on the use of GPU.
Training neural networks with capable GPU(s) is particularly helpful in cutting down the time it takes for training, so it is an absolute necessity to be able to do so using GPU. The following code assesses if GPU is used when training RNN model. Otherwise, it is highly recommended to use.
Why do we need GPU?
GPU stands for Graphical Processing Unit. It was originally designed to accelerate the rendering of 3D graphics. Over time, they became more flexible and programmable, enhancing their capabilities. Many graphics programmers leverage on this to create more interesting visual effects and realistic scenes with advanced lighting and shadowing techniques. Whilst, in our specific case, we tap onto the power of GPUs to dramatically accelerate additional workloads in high performance computing (HPC) and deep learning like in our model below.
# First checking if GPU is available
train_on_gpu=torch.cuda.is_available()
if(train_on_gpu):
print('Training on GPU.')
else:
print('No GPU available, training on CPU.')
# check if CUDA is available
use_cuda = torch.cuda.is_available()
No GPU available, training on CPU.
Caution: Since I do not have GPU available, this means my codes will take days to complete.
There is a lot of research and active work happening to think of ways to accelerate cloud computing.
What exactly is Transfer Learning and why do we need to use this technique?
On the other hand, our task will be to train a convolutional neural network (CNN) that can identify objects in images. If images are insufficiently provided, it means we could not have enough learnings when building a neural network to obtain a high accuracy. Therefore, instead of building and training a CNN from scratch, we’ll use a pre-built and pre-trained model applying transfer learning.Transfer learning is the most popular approach in deep learning.
The basic premise of transfer learning is simple: take a model trained on a large dataset and transfer its knowledge to a smaller dataset. The idea is the convolutional layers extract general, low-level features that are applicable across images — such as edges, patterns, gradients — and the later layers identify specific features within an image such as eyes or wheels.
Thus, we can use a network trained on unrelated categories in a massive dataset (usually Imagenet) and apply it to our own problem because there are universal, low-level features shared between images.
A convolutional neural network (CNN) is a specific type of artificial neural network that uses perceptrons, a machine learning unit algorithm, for supervised learning, to analyze data. CNNs apply to image processing, natural language processing and other kinds of cognitive tasks. A convolutional neural network is also known as a ConvNet.
What is a pre-trained model?
On normal circumstances, we used a a pre-trained model to solve a similar machine learning problem. Instead of building a model from scratch to solve a similar problem, we use the model trained on other problem as a starting point.
While choosing a pre-trained model, one should be careful in their case. If the problem statement we have at hand is very different from the one on which the pre-trained model was trained – the prediction we would get would be very inaccurate.
In simple language VGG is a deep CNN used to classify images. The layers in VGG19 model are as follows:
Architecture
In summary, VGG19Net contains 8 CNN layers with 3 layers of pooling and 3 fully connected(FC) layers. This makes up 19 layers in VGG19Net.
# instantiate VGG19 model
model_transfer=models.vgg19(pretrained=True)
#print(model_transfer)
print('Special Attention on line below as this required modification before we can apply onto our model:')
print()
print(model_transfer.classifier[6])
Special Attention on line below as this required modification before we can apply onto our model: Linear(in_features=4096, out_features=1000, bias=True)
# Freeze training for all "features" layers
for param in model_transfer.parameters():
param.requires_grad = False
# add last linear layer (n_inputs -> 3 skin types)
# new layers automatically have requires_grad = True
num_features = model_transfer.classifier[6].in_features
#model_transfer.classifier[2] = nn.Dropout(0.30)
model_transfer.classifier[3] = nn.Linear(num_features, 2048, bias=True)
#model_transfer.classifier[5] = nn.Dropout(0.30)
model_transfer.classifier[6] = nn.Linear(2048, num_skins, bias=True)
#model_transfer.classifier[6] = nn.Linear(num_features, num_skins, bias=True)
# move model to GPU if CUDA is available
if use_cuda:
model_transfer = model_transfer.cuda()
print('This is to review if the changes have been made accordingly.')
print('The last layer is now updated to:')
print(model_transfer.classifier[6])
print()
print('Fully Modified CNN Model using Transfer Learning:')
print(model_transfer)
This is to review if the changes have been made accordingly.
The last layer is now updated to:
Linear(in_features=2048, out_features=3, bias=True)
Fully Modified CNN Model using Transfer Learning:
VGG(
(features): Sequential(
(0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(1): ReLU(inplace=True)
(2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(3): ReLU(inplace=True)
(4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(6): ReLU(inplace=True)
(7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(8): ReLU(inplace=True)
(9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(11): ReLU(inplace=True)
(12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(13): ReLU(inplace=True)
(14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(15): ReLU(inplace=True)
(16): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(17): ReLU(inplace=True)
(18): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(19): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(20): ReLU(inplace=True)
(21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(22): ReLU(inplace=True)
(23): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(24): ReLU(inplace=True)
(25): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(26): ReLU(inplace=True)
(27): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(29): ReLU(inplace=True)
(30): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(31): ReLU(inplace=True)
(32): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(33): ReLU(inplace=True)
(34): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(35): ReLU(inplace=True)
(36): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
(avgpool): AdaptiveAvgPool2d(output_size=(7, 7))
(classifier): Sequential(
(0): Linear(in_features=25088, out_features=4096, bias=True)
(1): ReLU(inplace=True)
(2): Dropout(p=0.5, inplace=False)
(3): Linear(in_features=4096, out_features=2048, bias=True)
(4): ReLU(inplace=True)
(5): Dropout(p=0.5, inplace=False)
(6): Linear(in_features=2048, out_features=3, bias=True)
)
)
In most learning networks, error is calculated as the difference between the actual output y and the predicted output ŷ. The function that is used to compute this error is known as Loss Function also known as Cost function.
In my model, am using Cross-Entropy Loss (or Log Loss) function as this is a categorical classification problem.
Cross entropy measures the divergence between two probability distribution, if the cross entropy is large, which means that the difference between two distribution is large, while if the cross entropy is small, which means that two distribution is similar to each other.
from IPython.display import Image
Image("images/cross_entropy.png")
Note: P is the distribution of the true labels, and Q is the probability distribution of the predictions from the model.
On the other hand, optimizer is used to minimize the prediction error or loss.
The model while experiencing the examples of the training set, updates the model parameters weight,W. These error calculations when plotted against the W is also called cost function plot J(w), since it determines the cost/penalty of the model. So minimizing the error is also called as minimization the cost function.
But how exactly do you do that? Using optimizers. Optimizers are used to update weights and biases i.e. the internal parameters of a model to reduce the error. The most important technique and the foundation of how we train and optimize our model is using Gradient Descent.
In my case, I am using stochastic gradient descent (SGD). Stochastic Gradient Descent updates the parameters using only a single training instance in each iteration. The training instance is usually selected randomly. Stochastic gradient descent is often preferred to optimize cost functions when there are hundreds of thousands of training instances or more, as it will converge more quickly than batch gradient descent.
criterion_transfer = nn.CrossEntropyLoss()
optimizer_transfer = optim.SGD(model_transfer.classifier.parameters(), lr=0.001)
So far, we have loaded the images, flatten all images to Tensor dataset, modified our pre-trained VGG19Net model, specified loss function and optimizer to be applied. Next, we are ready to run and train the model.
# Workspaces automatically disconnect when the connection is inactive for about 30 minutes,
# which includes inactivity while deep learning models are training.
# You can use the workspace_utils.py module here to keep your connection alive during training.
import signal
from contextlib import contextmanager
import requests
DELAY = INTERVAL = 4 * 60 # interval time in seconds
MIN_DELAY = MIN_INTERVAL = 2 * 60
KEEPALIVE_URL = "https://nebula.udacity.com/api/v1/remote/keep-alive"
TOKEN_URL = "http://metadata.google.internal/computeMetadata/v1/instance/attributes/keep_alive_token"
TOKEN_HEADERS = {"Metadata-Flavor":"Google"}
def _request_handler(headers):
def _handler(signum, frame):
requests.request("POST", KEEPALIVE_URL, headers=headers)
return _handler
def active_session(delay=DELAY, interval=INTERVAL):
token = requests.request("GET", TOKEN_URL, headers=TOKEN_HEADERS).text
headers = {'Authorization': "STAR " + token}
delay = max(delay, MIN_DELAY)
interval = max(interval, MIN_INTERVAL)
original_handler = signal.getsignal(signal.SIGALRM)
try:
signal.signal(signal.SIGALRM, _request_handler(headers))
signal.setitimer(signal.ITIMER_REAL, delay, interval)
yield
finally:
signal.signal(signal.SIGALRM, original_handler)
signal.setitimer(signal.ITIMER_REAL, 0)
def keep_awake(iterable, delay=DELAY, interval=INTERVAL):
with active_session(delay, interval): yield from iterable
from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True
# number of epochs to train the model
n_epochs = 20
# train the model
def train(n_epochs, loaders, model, optimizer, criterion, use_cuda, save_path):
# initialize tracker for minimum validation loss
valid_loss_min = np.Inf
for epoch in range(1, n_epochs+1):
# initialize variables to monitor training and validation loss
train_loss = 0.0
train_correct = 0.
train_total = 0.
valid_loss = 0.0
valid_correct = 0.
valid_total = 0.
###################
# train the model #
###################
model.train()
for batch_idx, (data, target) in enumerate(loaders['train']):
# move to GPU
if use_cuda:
data, target = data.cuda(), target.cuda()
# clear the gradients of all optimized variables
optimizer.zero_grad()
# forward pass: compute predicted outputs by passing inputs to the model
output = model_transfer(data)
# calculate the loss
loss = criterion(output, target)
# backward pass: compute gradient of the loss with respect to model parameters
loss.backward()
# perform a single optimization step (parameter update)
optimizer.step()
## record the average training loss, using something like
train_loss = train_loss + ((1 / (batch_idx + 1)) * (loss.data - train_loss))
train_pred = output.data.max(1, keepdim=True)[1]
train_correct += np.sum(np.squeeze(train_pred.eq(target.data.view_as(train_pred))).cpu().numpy())
train_total += data.size(0)
train_acc = 100. * train_correct / train_total
######################
# validate the model #
######################
model.eval()
for batch_idx, (data, target) in enumerate(loaders['valid']):
# move to GPU
if use_cuda:
data, target = data.cuda(), target.cuda()
# forward pass: compute predicted outputs by passing inputs to the model
output = model_transfer(data)
# calculate the loss
loss = criterion(output, target)
## update the average validation loss
valid_loss = valid_loss + ((1 / (batch_idx + 1)) * (loss.data - valid_loss))
# calculate Validation Accuracy
# convert output probabilities to predicted class
pred = output.data.max(1, keepdim=True)[1]
# compare predictions to true label
valid_correct += np.sum(np.squeeze(pred.eq(target.data.view_as(pred))).cpu().numpy())
valid_total += data.size(0)
valid_acc = 100. * valid_correct / valid_total
# print training/validation statistics
print('Epoch: {} \tBatch: {} \tTrain Loss: {:.4f} \tTrain Acc: {:.3f} \tValid Loss: {:.4f} \tValid Acc: {:.3f}'.format(
epoch,
batch_idx,
train_loss,
train_acc,
valid_loss,
valid_acc
))
## TODO: save the model if validation loss has decreased
# save model if validation loss has decreased
if valid_loss <= valid_loss_min:
print('Validation loss decreased ({:.6f} --> {:.6f}). Saving model ...'.format(
valid_loss_min,
valid_loss))
torch.save(model.state_dict(), save_path)
valid_loss_min = valid_loss
# return trained model
return model
# train the model
model_transfer = train(n_epochs, data_loaders, model_transfer, optimizer_transfer,
criterion_transfer, use_cuda, 'model_transfer.pt')
# load the model that got the best validation accuracy (uncomment the line below)
model_transfer.load_state_dict(torch.load('model_transfer.pt'))
Epoch: 1 Train Loss: 0.780178 Train Acc: 68.65 Valid Loss: 0.953748 Valid Acc: 52.666666666666664 Epoch: 2 Train Loss: 0.770533 Train Acc: 68.8 Valid Loss: 0.950355 Valid Acc: 54.0 Epoch: 3 Train Loss: 0.756220 Train Acc: 68.85 Valid Loss: 0.958787 Valid Acc: 54.0 Epoch: 4 Train Loss: 0.756211 Train Acc: 69.0 Valid Loss: 0.974129 Valid Acc: 54.0 Epoch: 5 Train Loss: 0.754353 Train Acc: 68.6 Valid Loss: 0.984391 Valid Acc: 55.333333333333336 Epoch: 6 Train Loss: 0.756679 Train Acc: 68.5 Valid Loss: 0.958681 Valid Acc: 54.666666666666664 Epoch: 7 Train Loss: 0.746352 Train Acc: 69.3 Valid Loss: 0.922287 Valid Acc: 58.0 Epoch: 8 Train Loss: 0.736404 Train Acc: 69.25 Valid Loss: 0.944212 Valid Acc: 56.666666666666664 Epoch: 9 Train Loss: 0.745257 Train Acc: 69.35 Valid Loss: 0.958232 Valid Acc: 54.666666666666664 Epoch: 10 Train Loss: 0.736409 Train Acc: 69.3 Valid Loss: 0.922573 Valid Acc: 58.666666666666664 Epoch: 11 Train Loss: 0.739458 Train Acc: 68.9 Valid Loss: 0.931150 Valid Acc: 55.333333333333336 Epoch: 12 Train Loss: 0.727740 Train Acc: 69.95 Valid Loss: 0.933949 Valid Acc: 60.0 Epoch: 13 Train Loss: 0.738720 Train Acc: 69.75 Valid Loss: 0.957657 Valid Acc: 54.666666666666664 Epoch: 14 Train Loss: 0.740606 Train Acc: 69.25 Valid Loss: 0.920216 Valid Acc: 54.666666666666664 Epoch: 15 Train Loss: 0.729840 Train Acc: 69.2 Valid Loss: 0.941822 Valid Acc: 56.666666666666664 Epoch: 16 Train Loss: 0.724104 Train Acc: 70.4 Valid Loss: 0.926229 Valid Acc: 56.666666666666664 Epoch: 17 Train Loss: 0.727780 Train Acc: 69.7 Valid Loss: 0.989754 Valid Acc: 55.333333333333336 Epoch: 18 Train Loss: 0.731464 Train Acc: 69.4 Valid Loss: 0.885982 Valid Acc: 58.0 Epoch: 19 Train Loss: 0.724160 Train Acc: 69.9 Valid Loss: 0.946257 Valid Acc: 56.0 Epoch: 20 Train Loss: 0.721504 Train Acc: 69.3 Valid Loss: 0.936363 Valid Acc: 58.0 Validation loss decreased (inf --> 0.936363). Saving model ...
<All keys matched successfully>
# compare train loss vs valid loss
#fig, ax = plt.subplots(figsize=(12,8))
#plt.plot(train_loss, label='Training Loss', alpha=0.5)
#plt.plot(valid_loss, label='Validation Loss', alpha=0.5)
#plt.title("Training Loss vs Validation Loss Comparison")
#plt.legend()
After mentioned earlier, it took us 2-3 days to fully complete the model above. Now that it is completed, let's run the model on some independent test images and gauge its accuracy rate.
def test(loaders, model, criterion, use_cuda):
# monitor test loss and accuracy
test_loss = 0.
correct = 0.
total = 0.
model.eval()
for batch_idx, (data, target) in enumerate(loaders['test']):
# move to GPU
if use_cuda:
data, target = data.cuda(), target.cuda()
# forward pass: compute predicted outputs by passing inputs to the model
output = model(data)
# calculate the loss
loss = criterion(output, target)
# update average test loss
test_loss = test_loss + ((1 / (batch_idx + 1)) * (loss.data - test_loss))
# convert output probabilities to predicted class
pred = output.data.max(1, keepdim=True)[1]
# compare predictions to true label
correct += np.sum(np.squeeze(pred.eq(target.data.view_as(pred))).cpu().numpy())
total += data.size(0)
print('Test Loss: {:.6f}\n'.format(test_loss))
print('Test Accuracy: %2d%% (%2d/%2d)' % (100. * correct / total, correct, total))
# call test function
test(data_loaders, model_transfer, criterion_transfer, use_cuda)
Test Loss: 0.807614 Test Accuracy: 66% (397/600)
model_transfer.eval()
for batch_idx, (data, target) in enumerate(data_loaders['test']):
# forward pass: compute predicted outputs by passing inputs to the model
test_output = model_transfer(data)
We reached an accuracy of 66% But how can we improve further. As mentioned earlier, I am using CPU to run this neural network. Which means I am facing some level of computational resource issue. One way to improve my model further is to expand the number of iterations / num_epochs to 250. With this, we should be able to improve the accuracy further.
import torchvision.transforms as transforms
transforms = transforms.Compose([transforms.Resize(255),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])
# data directory / datasets
data_dir = './data'
test = datasets.ImageFolder(os.path.join(data_dir, 'test/'), transform=transforms)
#defining the data loaders
# the data is splitted between training, testing and validation sets
# data loader
loader = torch.utils.data.DataLoader(test, batch_size=20, num_workers=0, shuffle=False)
f_loader = {'test': loader}
print(f_loader)
{'test': <torch.utils.data.dataloader.DataLoader object at 0x0000017987186910>}
def sigmoid(x):
return 1 / (1 + np.exp(-x))
import tensorflow as tf
correct = 0
total = 0
pred = []
prob = []
tensor_list = []
model_transfer.eval()
for batch_idx, (data, target) in enumerate(data_loaders['test']):
output = model_transfer(data)
# convert output probabilities to predicted class
tensor_list.append(output)
pred.append(output.data.max(1, keepdim=True)[1])
prob.append(sigmoid(output.data))
# compare predictions to true label
#correct += np.sum(np.squeeze(pred.eq(target.data.view_as(pred))).cpu().numpy())
#total += data.size(0)
#print(target)
#print(pred.T)
#print(prob)
#print(t_output)
#print(t_output.data)
#print(t_prob)
import tensorflow as tf
a = tf.stack(prob)
a
<tf.Tensor: shape=(3,), dtype=float32, numpy=array([0.33051085, 0.673538 , 0.38840184], dtype=float32)>
p_Melanoma = []
p_Nevusha = []
p_SK = []
for each in n:
for prob in each:
p_Melanoma.append(prob[0])
p_Nevusha.append(prob[1])
p_SK.append(prob[2])
print("Length of Test Dataset:\nMelanoma: {} \tNevusha: {} \tSK: {}".format(len(p_Melanoma), len(p_Nevusha), len(p_SK)))
Length of Test Dataset: Melanoma: 600 Nevusha: 600 SK: 600
Hurray!, we have come this far. Looks like it, our model has 66% accuracy rate. This is by far as good as a normal dermatologist would assess in any given case.
from IPython.display import Image
Image(url='https://video.udacity-data.com/topher/2018/November/5be21b06_dancing-beemo/dancing-beemo.gif')
Now that the model has been trained and tested on independent images. Let's take a look at any specific image and see how we can use the model to learn about the some specific images.
display_image('./data/inference/melanoma/ISIC_9999999.jpg')
**image.resize((550,550), resample=Image.BICUBIC)**
The function used here has an optional resample parameter, that sets the resampling filter.
It can be NEAREST, BOX, BILINEAR, HAMMING, BICUBIC or LANCZOS. These are similar to the options you might see in imaging application like Photoshop or GIMP.
The list of filters is ordered from lowest to highest quality, but the higher quality options tend to be a bit slower. BICUBIC is probably the best option for general use (the default is NEAREST, but a modern computer can run BICUBIC pretty quickly and the quality improvement is quite noticeable).
Also, resize also has an optional parameter box that specifies a region to be resized. The image is first cropped to the box, and the result is resized. The crop box is specified by a 4-tuple in exactly the same way as the crop function below.
from PIL import Image
fig = plt.figure(figsize=(18,18))
fig.subplots_adjust(wspace=0.2, hspace=0.4)
plt.suptitle('External Samples Sourced Online|| Ref: Type of Skin Diseases\Image_File#', fontsize=20)
# samples some images
k = 0 #start with kth image
n = 3 # number of images required in each skin diseases #can go in a range of [2,3,4,5,6]
i = 0 # counter
for filename in np.hstack((np.array(glob("./data/inference/melanoma/*")),np.array(glob("./data/inference/nevus/*")),np.array(glob("./data/inference/seborrheic_keratosis/*")))):
i += 1
ax = fig.add_subplot(n,n,i)
image = Image.open(filename)
resize_image = image.resize((550,550), resample=Image.LANCZOS)
ax.set_ylim(512, 0)
ax.set_xlim(512, 0)
plt.title(filename.rsplit(".",1)[0].replace('./data/inference/','').replace('ISIC_',''))
ax.imshow(resize_image)
ax.set_axis_off()
ax.grid(None)
image = './data/inference/melanoma/ISIC_9999999.jpg'
feature_tensor = torch.from_numpy(np.array(Image.open(image).resize((224,224))))
print('Feature_Tensor shape:',feature_tensor.shape)
# reshape to channel first:
tensor_image = feature_tensor.view(feature_tensor.shape[2], feature_tensor.shape[0], feature_tensor.shape[1])
print('Tensor_Image shape:',tensor_image.shape)
tensor_image
Feature_Tensor shape: torch.Size([224, 224, 3]) Tensor_Image shape: torch.Size([3, 224, 224])
tensor([[[231, 158, 143, ..., 126, 232, 143],
[127, 224, 143, ..., 146, 131, 224],
[149, 132, 227, ..., 221, 144, 124],
...,
[ 98, 88, 174, ..., 227, 151, 135],
[235, 163, 144, ..., 109, 201, 123],
[111, 201, 123, ..., 87, 78, 171]],
[[ 95, 85, 181, ..., 227, 150, 134],
[212, 144, 123, ..., 152, 224, 148],
[136, 206, 129, ..., 63, 52, 145],
...,
[ 67, 180, 90, ..., 131, 118, 220],
[147, 134, 208, ..., 210, 135, 116],
[219, 153, 132, ..., 113, 182, 93]],
[[ 85, 167, 78, ..., 153, 140, 203],
[129, 116, 204, ..., 212, 135, 117],
[230, 153, 135, ..., 98, 166, 79],
...,
[235, 169, 150, ..., 143, 238, 164],
[145, 235, 160, ..., 143, 124, 217],
[136, 115, 220, ..., 242, 165, 145]]], dtype=torch.uint8)
print('Image(s) is accessible: ',os.path.exists('./data/inference'))
inf = np.array(glob("./data/inference/*/*"))
# print number of images in each dataset
print('There are %d ad-hoc images.' % len(inf))
Image(s) is accessible: True There are 3 ad-hoc images.
inf_data = datasets.ImageFolder(os.path.join(data_dir, 'inference/'), transform=test_transforms)
inf_data
Dataset ImageFolder
Number of datapoints: 3
Root location: ./data\inference/
StandardTransform
Transform: Compose(
Resize(size=255, interpolation=PIL.Image.BILINEAR)
CenterCrop(size=(224, 224))
ToTensor()
Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
)
batch_size_R = 10
#defining the data loaders
# the data is splitted between training, testing and validation sets
# data loader
inf_loader = torch.utils.data.DataLoader(inf_data, batch_size=batch_size_R, num_workers=0, shuffle=False)
inf_loaders = {'inf': inf_loader}
print(inf_loaders)
{'inf': <torch.utils.data.dataloader.DataLoader object at 0x000001798C2E3610>}
#for batch_idx, (data, target) in enumerate(inf_loaders['inf']):
# print(batch_idx,data,target)
model_transfer.eval()
for batch_idx, (data, target) in enumerate(inf_loaders['inf']):
# forward pass: compute predicted outputs by passing inputs to the model
output = model_transfer(data)
# convert output probabilities to predicted class
pred = output.data.max(1, keepdim=True)[1]
correct = (np.squeeze(pred.eq(target.data.view_as(pred))).cpu().numpy())
prob = sigmoid(output.data)
print(target, pred.T, correct)
print(tensor_list)
print(output.data)
print(prob)
tensor([1, 2, 3]) tensor([[1, 2, 1]]) [ True True False]
[]
tensor([[-1.2995, 1.5736, -0.3940],
[-1.4791, 0.3685, 0.7756],
[-0.7423, 2.1928, -1.3152]])
tensor([[0.2143, 0.8283, 0.4028],
[0.1856, 0.5911, 0.6847],
[0.3225, 0.8996, 0.2116]])
batch_size=20
train_loader1 = torch.utils.data.DataLoader(train_data, batch_size=batch_size, num_workers=0, shuffle=False)
valid_loader1 = torch.utils.data.DataLoader(valid_data, batch_size=batch_size, num_workers=0, shuffle=False)
test_loader1 = torch.utils.data.DataLoader(test_data, batch_size=batch_size, num_workers=0, shuffle=False)
data_loaders1 = {'train': train_loader1,'valid': valid_loader1,'test': test_loader1}
print(data_loaders1)
for batch_idx, (data, target) in enumerate(data_loaders1['valid']):
if batch_idx < 15:
print(target)
{'train': <torch.utils.data.dataloader.DataLoader object at 0x000001798BED0400>, 'valid': <torch.utils.data.dataloader.DataLoader object at 0x000001798BED0460>, 'test': <torch.utils.data.dataloader.DataLoader object at 0x000001798BED0520>}
tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
tensor([1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])
tensor([2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])
tensor([2, 2, 2, 2, 2, 2, 2, 2, 2, 2])
Once we trained and tested our model, we then create a CSV file to store our test predictions. Note, our test file should have exactly 600 rows, each corresponding to a different test image, plus a header row.
Our file should have exactly 3 columns in the csv file.
Samples of output should look like this:
Id,task_1,task_2
data/test/melanoma/ISIC_0013242.jpg,0,3.88E-33
data/test/melanoma/ISIC_0013321.jpg,0,2.41E-12
data/test/melanoma/ISIC_0013411.jpg,0,1.62E-31
data/test/nevus/ISIC_0016029.jpg,0,0
data/test/nevus/ISIC_0016030.jpg,7.62E-32,8.81E-28
data/test/nevus/ISIC_0016031.jpg,1.10E-14,7.91E-16
data/test/seborrheic_keratosis/ISIC_0012848.jpg,0,0
data/test/seborrheic_keratosis/ISIC_0012522.jpg,0,5.84E-33
data/test/seborrheic_keratosis/ISIC_0014386.jpg,0,1.65E-25
Before we run the probability, let's make sure there are 600 images in the test data folder.
print('There are {} batches which consists of a total of {} images.'.format(batch_size, batch_size * len(data_loaders['test'])))
print('The breakdown of images are as follows:')
There are 20 batches which consists of a total of 600 images. The breakdown of images are as follows:
for filename in np.hstack((test_folder)):
print(filename + 'has:{} images.'.format(len(np.array(glob(filename+'\*')))))
./data/test\melanomahas:117 images. ./data/test\nevushas:393 images. ./data/test\seborrheic_keratosishas:90 images.
name = []
i = 0
for filename in np.hstack(np.array(glob("./data/test/*/*"))):
i+=+1
name.append(filename.replace('./data/test\\','data/test/').replace('\\','/'))
df = pd.DataFrame(columns = ['Id', 'task_1', 'task_2'])
df['Id'] = name
df['task_1'] = p_Melanoma
df['task_2'] = p_SK
df.head()
| Id | task_1 | task_2 | |
|---|---|---|---|
| 0 | data/test/melanoma/ISIC_0012258.jpg | 0.516 | 0.232 |
| 1 | data/test/melanoma/ISIC_0012356.jpg | 0.315 | 0.348 |
| 2 | data/test/melanoma/ISIC_0012369.jpg | 0.556 | 0.177 |
| 3 | data/test/melanoma/ISIC_0012395.jpg | 0.382 | 0.333 |
| 4 | data/test/melanoma/ISIC_0012425.jpg | 0.467 | 0.317 |
df.tail()
| Id | task_1 | task_2 | |
|---|---|---|---|
| 595 | data/test/seborrheic_keratosis/ISIC_0014647.jpg | 0.313 | 0.206 |
| 596 | data/test/seborrheic_keratosis/ISIC_0014648.jpg | 0.265 | 0.311 |
| 597 | data/test/seborrheic_keratosis/ISIC_0014649.jpg | 0.485 | 0.247 |
| 598 | data/test/seborrheic_keratosis/ISIC_0014652.jpg | 0.615 | 0.376 |
| 599 | data/test/seborrheic_keratosis/ISIC_0014653.jpg | 0.331 | 0.388 |
df.to_csv('my_prediction.csv',index=False)